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15 December 1972 


MEMORANDUM FOR: 


SUBJECT : Proposal Concerning Consumer Evaluation 

of NIEs and SNIEs 



1. Attached, as you requested in your memorandum of 14 December, 
is my suggestion for a draft proposal for Mr. Tweedy to the NSCIC 
Working Group re consumer evaluation of NIEs and SNIEs. 

2. First, a few words of explanation of why I came out where 
I did. 


-- I gather from your memorandum we have accepted 
Huizenga's position that the NSCIC Working Group not be 
involved in the scheduling of NIEs. (As you recall, he 
expressed the belief that it is necessary to keep intel- 
ligence and policy separate -- "Intelligence must be free 
to produce what it thinks is useful and needed."). As a 
result, my proposal does not include this facet of our 
original suggestion. 

-- In your memorandum to me you suggested that PRG 
might prepare summaries and commentaries for the WG and 
BNE from the returned questionnaires. Because, I feel, 
this could give BNE something more to object to -- charges of 
PRG misrepresenting comments or stressing the wrong confluents 
-- my proposal is slightly different. 

-- Finally, I believe my proposal will involve more 
direct WG consumer participation in the NIE/SNIE review 
process, and it will allow for discussion in depth of the 
most important (or typical) estimates. It may also provide 
a precedent according to which the WG can evaluate other 
types of intelligence -- memoranda, current periodical 
reporting, and NSSM annexes. 
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Dist: 

Original - Adse. 

1 - PRG Chrono 
1 - I I Chrono 
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DRAFT 

0100003-® December 7 972 


MEMORANDUM FOR: NSCIC Working Group Members 

SUBJECT : Proposal Concerning Consumer Evaluation 

of NIEs and SNIEs 

1. As another step in responding to the Presidential memorandum 

of 5 November 1971 to the DCI in which he called for improvement in 

the "quality, scope and timeliness of the community's product," it is 
proposed that the Working Group undertake a program of systematic 
assessment of the strengths, weaknesses and general usefulness of National 

Intelligence Estimates and Special National Intelligence Estimates. 

2. Although there always has been feedback from recipients of 
specific NIEs and SNIEs to their producers and there have been several 
intensive efforts to ascertain consumer reaction to estimates, in general 
the intelligence community has never before had a forum in which a 
broad spectrum of high level consumers can voice their view with 
respect to specific estimates or specific types of estimates. The 
NSCIC Working Group is such a forum. The proposal which follows out- 
lines a procedure by which the reactions of important consumers can be 
presented directly to the Chairman, BNE and other producer members of 
the Working Group. 

3. It is proposed that the Working Group select from the attached 
list of estimates produced in the last six months four NIEs/SNIEs which 
consumers will be prepared at our next meeting to critique in depth 
from the points of view of their departments. Attached also is a 
sample questionnaire which is designed to aid each WG consumer in 
eliciting opinions concerning these estimates from users within 

his organization if he should so desire as an aid in formulating his 
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own response. At our next meeting, then, we would have a frank 
discussion of what part these specific estimates played or are 
expected to play in the formulation and execution of U.S. policy. 

We should know what are the favorable and unfavorable aspects of 
the estimates and we should have suggestions on how such estimates 
could be improved in the future. 

4. Depending on the lessons learned from the first critiques, 

I would propose that the Working Group endeavor to assess a selected 
group of estimates each quarter as a continuing process. 


Bronson Tweedy 

Chairman, NSCIC Working Group 


Attachment: 
As stated 
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EVALUATION REPORT 




A. Identification of Estimate Evaluated 

(NIE) (SNIE) No. 

Ti tl e 

B. Source of Evaluation 

Organization address: 


Contact: 

Name Telephone 

' • ; 

C. Estimate Utility and Timeliness . • 

1. Primary use to which estimate is being or will be put: - 

a. Policy planning d. Military force planning 

b. Operations planning e. Military R&D planning 

c. Background information f. Other (specify) 

2. Utility of this estimate for ongoing or anticipated policy/operations 
problems: 

a. Absolute necessity d. Nice to have , 

b. Very useful e. Little or no use ( 

c. Useful -i 


3’. If some other source currently satisfies the need for this 

estimative information in an equal or better fashion, identify 
such source: 


4. Timeliness of the estimate: 

a. Very timely b. Timely c. Not timely 

If c. checked, how could timeliness have been improved? 

Why is estimate not considered timely? 


1 
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F. Expression of Uncertainties 

• 1. The uncertainties relating to the data or the situations 

described are brought out: 

a. Excellently b. Satisfactorily c. Inadequately 

2. For purposes of your organization, the estimate treats the 
full range of uncertainties: 

a. Very well b. Adequately c. Inadequately 

3. How could the treatment of uncertainty be Improved? 
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G. Expression of Alternatives 

1. For purposes of your organization, the range of alternative 
projections and the description of each alternative are: 


a. Quite comprehensive 


b. Adequate 


c. Inadecuate 


2. How could the presentation of alternatives be improved? 


Quality of the Estimative Logic 

* » 
1. The extent to which the analysis logically ties together 
the factual and estimative elements of the paper i's: ' 


a. Excellent 


b. Satisfactory 


c. Inadequate 


2. How might this aspect of the estimate be improved? 


I. Overall Reaction to the Estimate 

(NOTE: -The purpose of this section is to encourage you to summarize 
your reactions to the estimate and to present such additional comments 
as you consider useful as feedback to the intelligence community. If 
you have made hand-written notes on pages of the estimate itself which 
you consider would be of use to those who produced it, it would be 
appreciated if the estimate is returned with the evaluation form.) 

1. Which of the following words and phrases characterize this 
estimate: 


Exceptionally valuable 

Stimulating and persuasive 
Imaginative new viewpoint ~ 


Rehash of conventional 
wisdom 

Shallow and unconvincing 
Dull 
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All wo would no :d 
in a • .dtion to this 
would Tdc a list of 
estimates producod 
in tho last six 
months. 

I would sugg. st that 
that tho questionnaire 
wo proposed before and 
is attached would bo 
adequate fon/ our 
purposes. 
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